Recorded data processing method and recorded data processing device

ABSTRACT

A recorded data processing method includes: for each of N recorded data pairs each formed by two pieces of recorded data X that are adjacent to each other when N pieces of recorded data X each representing a recording target including at least one of audio and video are arranged cyclically, calculating a plurality of candidate values for a time difference between time signals representing temporal changes of the recording target in the two respective pieces of recorded data X of the recorded data pair; and identifying one of the plurality of candidate values in each of the N recorded data pairs as the time difference between the two pieces of recorded data in the recorded data pair such that a numerical value obtained by summing, over the N recorded data pairs, one of the plurality of candidate values calculated for each of the N recorded data pairs approaches zero.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part application of InternationalApplication No. PCT/JP2017/008267, filed Mar. 2, 2017, which claimspriority to Japanese Patent Application No. 2016-045131 filed in Japanon Mar. 9, 2016. The entire disclosures of International Application No.PCT/JP2017/008267 and Japanese Patent Application No. 2016-045131 arehereby incorporated herein by reference.

BACKGROUND Technological Field

The present technology relates to a processing recorded data.

Background Art

Various technologies have been proposed for processing mutual temporalrelation between a plurality of pieces of recorded data including videoand audio. For example, Japanese Laid-Open Patent Application No.2008-193561 (hereinafter referred to as Patent Document 1) disclosestechnology for analyzing a plurality of pieces of audio data eachrecorded at a time of capturing of a plurality of videos obtained bycapturing a same subject from different positions, and thereby generatestime difference information for synchronizing the plurality of videos.Specifically, the time difference information is generated according toa shift time that maximizes a cross-correlation function between twopieces of audio data.

SUMMARY

A recorded data processing method in accordance with some embodimentsincluding: calculating for each of N (N is a natural number of three ormore) recorded data pairs each formed by two pieces of recorded datathat are adjacent to each other when N pieces of recorded data eachrepresenting a recording target including at least one of audio andvideo are arranged cyclically, a plurality of candidate values for atime difference between time signals representing temporal changes ofthe recording target in the two respective pieces of recorded data ofthe recorded data pair; and identifying one of the plurality ofcandidate values in each of the N recorded data pairs as the timedifference between the two pieces of recorded data in the recorded datapair such that a numerical value obtained by summing, over the Nrecorded data pairs, one of the plurality of candidate values calculatedfor each of the N recorded data pairs approaches zero.

A recorded data processing device in accordance with some embodimentsincluding: a candidate calculating unit configured to calculate, foreach of N (N is a natural number of three or more) recorded data pairseach formed by two pieces of recorded data that are adjacent to eachother when N pieces of recorded data each representing a recordingtarget including at least one of audio and video are arrangedcyclically, a plurality of candidate values for a time differencebetween time signals representing temporal changes of the recordingtarget in the two respective pieces of recorded data of the recordeddata pair; and an analysis processing unit configured to identify one ofthe plurality of candidate values in each of the N recorded data pairsas the time difference between the two pieces of recorded data in therecorded data pair such that a numerical value obtained by summing, overthe N recorded data pairs, one of the plurality of candidate valuescalculated for each of the N recorded data pairs approaches zero.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a recorded data editing system according toa first embodiment.

FIG. 2 is a diagram of assistance in explaining time differences in Nrecorded data pairs.

FIG. 3 is a diagram of assistance in explaining the absolute value ofcross-correlation of a recorded data pair.

FIG. 4 is a diagram of assistance in explaining the cross-correlationafter smoothing.

FIG. 5 is a flowchart of a process for generating content by anelectronic controller.

FIG. 6 is a diagram of assistance in explaining time differences in Nrecorded data pairs according to a second embodiment.

FIG. 7 is a diagram of assistance in explaining time differences in adifferent permutation of the N recorded data pairs.

FIG. 8 is a flowchart of processing in which an electronic controllergenerates content.

DESCRIPTION OF EMBODIMENTS

In conventional systems, there is a possibility that an error betweenthe shift time calculated from the maximum value of thecross-correlation function and an actual shift time may be increasedwhen the audio data includes reverberation or noise, for example. Inview of the above circumstances, it is an object of some embodiments toidentify a time difference between pieces of recorded data with highaccuracy.

Selected embodiments will now be explained with reference to thedrawings. It will be apparent to those skilled in the audio field fromthis disclosure that the following descriptions of the embodiments areprovided for illustration only and not for the purpose of limiting theinvention as defined by the appended claims and their equivalents.

First Embodiment

FIG. 1 is a block diagram of a recorded data editing system 10 accordingto a first embodiment. The recorded data editing system 10 is a computersystem for processing audio (for example, voice or musical audio) andmovie. As illustrated in FIG. 1, the recorded data editing system 10includes an electronic controller 22, a storage device 24, acommunicating device 26, a display device 32, an audio emitting device34, and an operating device 36. In some embodiments, example systemarrangements for the recorded data editing system 10 include, but arenot limited to, a portable information processing device, a cellularphone, a smart phone, a tablet terminal, a personal computer, or thelike. Also, the recorded data editing system 10 can also be implementedby a stationary information processing device.

The term “electronic controller” as used herein refers to hardware thatexecutes software programs. The electronic controller 22 includesprocessing device (for example, a central processing unit (CPU)) thatcontrols the elements of recorded data editing system 10.

The term “communicating device” as used herein includes a receiver, atransmitter, a transceiver and a transmitter receiver, capable oftransmitting and/or receiving communication signals. In this embodiment,the communicating device 26 transmits communication signals, and thus,the communicating device 26 can be a transmitter, a transceiver or atransmitter receiver.

The communicating device 26 communicates with a plurality of (N)recording devices 12 (N is a natural number of three or more). Eachrecording device 12 is an apparatus including an audio collecting devicecollecting audio and a video device collecting video by capturing videoand audio. Each recording device 12 generates data X representing theaudio collected by the audio collecting device and the video collectedby the video device (which data will hereinafter be referred to as“recorded data”). The recording devices 12 are video apparatuses such asdigital camcorders or the like and information terminals such as cellarphones, smart phones, or the like having a recording function. Thecommunicating device 26 in the first embodiment receives recorded data Xfrom each of the N recording devices 12. Specifically, the communicatingdevice 26 receives the recorded data X from each of the recordingdevices 12 by publicly known short-range radio communication such asWi-Fi (registered trademark), Bluetooth (registered trademark), or thelike. However, a method of communication between the communicatingdevice 26 and each of the recording devices 12 is arbitrary. Forexample, the communicating device 26 can also communicate with each ofthe recording devices 12 by wire.

The storage device 24 is formed by recording medium such as a magneticrecording medium, a semiconductor recording medium, or the like. Thestorage device 24 stores a program executed by the electronic controller22 and various kinds of data used by the electronic controller 22. Thestorage device 24 in the first embodiment stores N pieces of recordeddata X1 to XN received by the communicating device 26 from therespective recording devices 12. It is also possible to store the Npieces of recorded data X1 to XN in the storage device 24 in advance. Inthis case, the communicating device 26 can be omitted from the recordeddata editing system 10. In addition, it is also possible to install thestorage device 24 in a server with which the recorded data editingsystem 10 can communicate (that is, a cloud storage). In this case, thestorage device 24 can be omitted from the recorded data editing system10.

The N recording devices 12, for example, record, in parallel with eachother, audio and video as a common recording target (recording object)at mutually different positions. For example, the plurality of recordingdevices 12 are arranged at mutually different positions in a commonacoustic space such as a hall, a concert hall, a dance hall or the like,and each of the plurality of recording devices 12 generates the recordeddata X by recording a state of a stage and audience, for example, from adifferent angle. The recorded data X in the first embodiment representsthe recording target including the audio collected by an audiocollecting device and the video collected by a video device.Specifically, the recorded data X includes an audio signal representingtemporal changes in the audio collected by the audio collecting deviceand a video signal representing temporal changes captured by the videodevice (that is, a movie). For example, in a case where the played audioof a musical piece for a play performed on a stage is reproduced from anaudio emitting device (for example, a speaker) installed on the stage,the audio of the recorded data X recorded by each of the recordingdevices 12 includes the played audio in common (though audiocharacteristics such as volume and the like can differ). A user of eachrecording device 12 separately starts recording by the own recordingdevice 12. Hence, a start point of recording of the audio and the videodoes not precisely coincide between the N pieces of recorded data X1 toXN, but can differ for each piece of recorded data X. That is, there aretime differences between the N pieces of recorded data X1 to XN.Incidentally, the first embodiment assumes, for convenience, a casewhere recording periods of all of the N pieces of recorded data X1 to XNpartly overlap each other on a time axis. Incidentally, while therecording target including both the audio and the video is illustrated,a recording target including only one of audio and video can also beassumed.

The display device 32 (for example, a liquid crystal display panel) inFIG. 1 displays videospecified from the electronic controller 22. Theaudio emitting device 34 (for example, a speaker or headphones) emitsaudio specified from the electronic controller 22. The operating device36 is an input apparatus receiving an instruction from a user. Theoperating device 36 is, for example, formed by a plurality of operatingelements detecting operations by the user or a touch panel detectingcontact of the user with a display surface of the display device 32.

The electronic controller 22 implements a plurality of functions (arecorded data analyzing unit 40 and an edit processing unit 46) forprocessing the N pieces of recorded data X1 to XN by executing theprogram stored in the storage device 24. Incidentally, it is alsopossible to adopt a configuration in which a part of the functions ofthe electronic controller 22 is implemented by an electronic circuitdedicated to audio processing or video processing or a configuration inwhich the functions of the electronic controller 22 are distributed to aplurality of devices.

As illustrated in FIG. 2, the recorded data analyzing unit 40 determinesa time difference Oij (i, j=1 to N, i≠j) between two pieces of recordeddata Xi and Xj that are adjacent to each other (next to each other) whenthe N pieces of recorded data X1 to XN generated by the recordingdevices 12 are arranged cyclically. The cyclic arrangement of the Npieces of recorded data X1 to XN means an arrangement (annulararrangement) in which the N pieces of recorded data X1 to XN arearranged in series with each other and the first recorded data X1 ismade to follow the last recorded data XN. Hence, the cyclic arrangementof the N pieces of recorded data X1 to XN includes a pair (hereinafterreferred to as a “recorded data pair”) Pij formed by two pieces ofrecorded data Xi and Xj adjacent to each other. That is, there are Ncombinations between the numerical value i and the numerical value j:(i, j)=(1, 2), (2, 3), . . . , (N-1, N), (N, 1). That is, thearrangement of the N pieces of recorded data X1 to XN includes Nrecorded data pairs P12 to PN1. As is understood from FIG. 2, the timedifference Oij means a relative time (offset) of the recorded data Xjwhen the recorded data Xi is set as a reference. Incidentally, thepermutation of the N pieces of recorded data X1 to XN arrangedcyclically is arbitrary.

As illustrated in FIG. 1, the recorded data analyzing unit 40 in thefirst embodiment includes a candidate value calculating unit 42 and ananalysis processing unit 44. For each of N recorded data pairs P12 toPN1 each formed by two pieces of recorded data Xi and Xj that areadjacent to each other when the N pieces of recorded data X1 to XN arearranged cyclically, the candidate value calculating unit 42 calculatesa plurality of candidate values for a time difference between audiosignals (an example of time signals) in the two respective pieces ofrecorded data Xi and Xj of the recorded data pair Pij. One of theplurality of candidate values calculated for the recorded data pair Pijis selected as a final time difference Oij.

Specifically, for each of the N recorded data pairs P12 to PN1, thecandidate value calculating unit 42 calculates a plurality of candidatevalues according to an audio signal cross-correlation Cij(τ) between therecorded data Xi and the recorded data Xj. In the first embodiment, theplurality of candidate values are calculated according to an absolutevalue |Cij(τ)| of the cross-correlation Cij(τ). As expressed by thefollowing Equation (1), the cross-correlation Cij(τ) is a numericalstring indicating a degree of time waveform correlation between an audiosignal yi(t) included in the recorded data Xi and an audio signal yj(t)included in the recorded data Xj with a time difference (amount of shifton the time axis) τ of the audio signal yj(t) to the audio signal yi(t)as a variable after a starting point of the audio signal yi(t) and astarting point of the audio signal yj(t) are made to coincide with eachother on the time axis. Incidentally, the time difference τ can assume anegative numerical value. Hence, for example, when the recorded data Xjis positioned in the rear of the recorded data Xi on the time axis, thetime difference Oij is a positive number, and when the recorded data Xjis positioned in front of the recorded data Xi on the time axis, thetime difference Oij is a negative number. Incidentally, the plurality ofcandidate values may be calculated without depending on the absolutevalue of the cross-correlation Cij(τ).

$\begin{matrix}\lbrack {{Math}.\mspace{14mu} 1} \rbrack & \; \\{{C_{ij}(\tau)} = {\frac{1}{N + 1}{\sum\limits_{t = 0}^{N}{{y_{i}(t)}{y_{j}( {t + \tau} )}}}}} & (1)\end{matrix}$

In addition, as expressed by Equation (2), the cross-correlation Cij(τ)can also be calculated by an inverse Fourier transform (IFFT) of a crossspectrum of a frequency spectrum Yi(f) of the audio signal yi(t) and afrequency spectrum Yj(f) of the audio signal yj(t). f denotes frequency.Yi*(f) is a complex conjugate of Yi(f). A configuration that calculatesthe cross-correlation Cij(τ) by operation of Equation (2) has anadvantage of being able to reduce an amount of calculation incalculating the cross-correlation Cij(τ) as compared with aconfiguration that calculates Equation (1).

[Math. 2]

C _(ij)(τ)=IFFT(Y* _(i)(f)Y _(j)(f))   (2)

FIG. 3 is a diagram of assistance in explaining the absolute value|Cij(τ)| of the cross-correlation Cij(τ) calculated for one arbitraryrecorded data pair Pij. The higher the correlation between the timewaveform of the audio signal yi(t) and the time waveform of the audiosignal yj(t), the larger the value that the absolute value |Cij(τ)| canassume. The recording periods of the recorded data X partly overlap eachother on the time axis, as described earlier. Thus, the audio includedin each piece of recorded data X includes an audio (performance audio ofa musical piece for stage performance) component partly common on thetime axis. Hence, a time difference τ maximizing the absolute value|Cij(τ)| of the cross-correlation Cij(τ) of the recorded data pair Pijcan be the time difference Oij of the corresponding recorded data. Inactuality, however, in a case where the audio included in the recordeddata X includes noise, for example, there is a possibility of an erroroccurring when the time difference τ maximizing the absolute value|Cij(τ)| of the cross-correlation Cij(τ) is determined as the timedifference Oij of the recorded data pair Pij. In order to solve thisproblem, the first embodiment adopts a configuration that identifies oneof a plurality of candidate values D calculated according to thecross-correlation Cij(τ) as the time difference Oij between the twopieces of recorded data Xi and Xj.

In calculating the plurality of candidate values D, specifically, thecandidate value calculating unit 42 smooths the absolute value |Cij(τ)|of the calculated cross-correlation Cij(τ). For smoothing the absolutevalue |Cij(τ)| of the calculated cross-correlation Cij(τ), for example,the candidate value calculating unit 42 includes a function of a movingaverage calculation. The candidate value calculating unit 42 identifiesM candidate values D (M is a natural number of two or more) fromcross-correlation Cij_s(τ) after the smoothing. FIG. 4 is a diagram ofassistance in explaining the cross-correlation Cij_s(τ) after thesmoothing. As illustrated in FIG. 4, the M candidate values D (D1, D2, .. . , DM) are a plurality of time differences τ corresponding to Mrespective maxima in the cross-correlation Cij_s(τ), and are a pluralityof candidate values for the time difference Oij of the recorded datapair Pij. The M maxima are, among a plurality of maxima, for example,maxima up to an Mth maximum in descending order of maximal values or Mmaxima whose maximal values exceed a threshold value. When the M maximaexceeding the threshold value are identified, a total number M ofcandidate values D can differ for each recorded data pair Pij.Incidentally, the threshold value is selected empirically orstatistically. The total number M of candidate values D is arbitrary.The smaller the total number of candidate values D is, the more aprocessing load on the electronic controller 22 can be reduced.

As illustrated in FIG. 3, the absolute value |Cij(τ)| of thecross-correlation Cij(τ) locally tends to be a large numerical value ina range around the time difference τ maximizing the absolute value|Cij(τ)|. Hence, if the M candidate values D of the time difference Oijare calculated in descending order of the absolute value |Cij(τ)| of thecross-correlation Cij(τ) without smoothing, in a case where the absolutevalue |Cij(τ)| of the cross-correlation Cij(τ) locally increases due,for example, to noise or the like, there is a possibility of a pluralityof candidate values D being locally identified from the vicinitythereof. In the first embodiment, as illustrated in FIG. 4, theplurality of time differences τ corresponding to the respective maximaof the cross-correlation Cij_s(τ) after smoothing are identified as aplurality of candidate values D. The plurality of candidate values D aretherefore dispersed. That is, even when the absolute value |Cij(τ)| ofthe cross-correlation Cij(τ) becomes a maximum due, for example, tonoise or the like, it is possible to identify the plurality of candidatevalues D so as to include a proper time difference Oij between the twopieces of recorded data Xi and Xj (in turn, identify the time differenceOij with high accuracy). Needless to say, when the localization of theplurality of candidate values D does not present a particular problem,the plurality of candidate values D can also be identified from theabsolute value |Cij(τ)| of the cross-correlation Cij(τ). As describedabove, the first embodiment calculates the plurality of candidate valuesD according to the cross-correlation Cij(τ) between the audio signalsy(t) of the recorded data pair Pij, and therefore has an advantage ofbeing able to simply calculate the plurality of candidate values D forthe time difference τ between the pieces of recorded data X as comparedwith a configuration that calculates the plurality of candidate values Daccording to an index other than the cross-correlation Cij(τ).

For each of the N recorded data pairs P12 to PN1, the analysisprocessing unit 44 in FIG. 1 identifies one of the M candidate values Das the time difference Oij between the two pieces of recorded data Xiand Xj in the recorded data pair Pij.

As is understood from FIG. 2, a total value S (S=O12+O23+ . . . +ON1) ofproper time differences O12 to ON1 over the N recorded data pairs P12 toPN1 is zero. Hence, a combination of candidate values D whose totalvalue S is close to zero can be evaluated as being close to actual timedifferences. That is, the total value S of N candidate values Didentified for the N respective recorded data pairs P12 to PN1 can beused as an index indicating a degree of reliability of the candidatevalues D.

Specifically, over the N recorded data pairs P12 to PN1, the analysisprocessing unit 44 calculates a total value S of N candidate values Dmfor all combinations in which one candidate value Dm is selected from Mcandidate values D of each recorded data pair Pij (that is, combinationsof N candidate values Dm), and identifies a combination of candidatevalues Dm whose total value S is closest to zero (that is, a combinationof candidate values Dm in which the absolute value of the total value Sthereof is a minimum). The analysis processing unit 44 determines the Nrespective candidate values Dm included in the identified combination astime differences O12 to ON1 of the N recorded data pairs P12 to PN1. Asis understood from the above description, the analysis processing unit44 functions as an element that identifies one of the plurality ofcandidate values D in each of the N recorded data pairs P12 to PN1 asthe time difference Oij between the two pieces of recorded data Xi andXj in the recorded data pair Pij such that a numerical value (totalvalue S) obtained by summing one of the plurality of candidate values Dcalculated for each of the N recorded data pairs P12 to PN1 over the Nrecorded data pairs P12 to PN1 approaches zero.

The edit processing unit 46 generates content Z in which the N pieces ofrecorded data X1 to XN are synchronized with each other according to theN time differences O12 to ON1 determined by the analysis processing unit44. The synchronization of the recorded data X means a state in whichthe time axes of audio and video of the respective pieces of recordeddata X are made to coincide with each other in the N pieces of recordeddata X1 to XN. Specifically, the edit processing unit 46 adjusts theposition on the time axis of each piece of recorded data X such that aspecific time in each piece of recorded data X is a common time point onthe time axis over the N pieces of recorded data X1 to XN. That is, asillustrated in FIG. 2, the position on the time axis of each piece ofrecorded data X is adjusted such that the time difference between therecorded data Xi and the recorded data Xj is the time difference Oijcalculated by the analysis processing unit 44.

The content Z generated by the edit processing unit 46 in FIG. 1 isreproduced according to an instruction of the electronic controller 22.Specifically, the video of the content Z is displayed by the displaydevice 32, and the audio of the content Z is emitted by the audioemitting device 34.

FIG. 5 is a flowchart of processing in which the electronic controller22 generates content Z. The processing of FIG. 5 is started by beingtriggered by an instruction from a user to the operating device 36. In acase where there are, for example, four pieces of recorded data X1 to X4obtained by recording a state of a play performed on a stage frommutually different positions, when the processing of FIG. 5 is started,the candidate value calculating unit 42 calculates the absolute value|Cij(τ)| of cross-correlation Cij(τ) between the audio signal yi(t) ofthe recorded data Xi and the audio signal yj(t) of the recorded data Xjfor each of four recorded data pairs P12, P23, P34, and P41 (SA1). Thecandidate value calculating unit 42 smooths the absolute value |Cij(τ)|of the cross-correlation Cij(τ) which absolute value is calculated foreach of the four recorded data pairs Pij (SA2). For eachcross-correlation Cij_s(τ) after the smoothing of the four recorded datapairs P12 to P41, the candidate value calculating unit 42 identifiesfive time differences τ corresponding to respective maxima up to a fifth(that is, M=5) maximum in descending order as five candidate values D(D1 to D5) (SA3). Steps SA1 to SA3 are processing of calculating theplurality of candidate values D.

Over the four recorded data pairs P12 to P41, the analysis processingunit 44 determines one arbitrary combination from all combinations inwhich one candidate value Dm is selected from the five candidate valuesD of each recorded data pair P (that is, combinations of four candidatevalues Dm) (SB1). The analysis processing unit 44 calculates a totalvalue S of the four candidate values Dm in the determined combination(SB2). The analysis processing unit 44 repeats the processing of stepSB1 and step SB2 until completing the calculation of total values S forall the combinations (SB3: NO). When calculating the total values S ofall the combinations (SB3: YES), the analysis processing unit 44determines the four candidate values Dm corresponding to a total value Sthat is closest to zero among the total values S of all the combinationsas respective time differences O12, O23, O34, and O41 of the fourrespective recorded data pairs P12 to P41 (SB4). Steps SB1 to SB4 areprocessing of identifying the respective time differences O12 to O41 ofthe four recorded data pairs P12 to P41.

The edit processing unit 46 generates content Z by edit processing thatsynchronizes the four pieces of recorded data X1 to X4 with each otheraccording to the time differences O12 to O41 identified by the analysisprocessing unit 44 (SC1).

As is understood from the above description, in the first embodiment,one of the plurality of candidate values D in each of the N recordeddata pairs P12 to PN1 is identified as the time difference Oij betweenthe two pieces of recorded data Xi and Xj in the recorded data pair Pijsuch that the total value S of candidate values Dm for the timedifferences τ over the N recorded data pairs P12 to PN1 approaches zero.Hence, the time difference Oij between the pieces of recorded data X canbe identified with high accuracy as compared with a configuration thatadopts the sole time difference τ identified from the two pieces ofrecorded data Xi and Xj as a final value (time difference Oij), forexample, a configuration that determines the time difference τmaximizing the time signal cross-correlation Cij(τ) between the twopieces of recorded data Xi and Xj as the time difference Oij between thepieces of recorded data X. That is, it is possible to reduce an errorbetween the time difference Oij between the pieces of recorded data Xand an actual time difference.

Second Embodiment

The second embodiment will be described. In each of the embodimentsillustrated in the following, elements that have the same actions orfunctions as in the first embodiment have been assigned the samereference symbols as those used to describe the first embodiment, andthe detailed descriptions thereof have been appropriately omitted.

FIG. 6 is a diagram of assistance in explaining N time differences O12to ON1 in N respective recorded data pairs P12 to PN1 according to thesecond embodiment. The first embodiment illustrates a case where therecording periods of all of the N pieces of recorded data X1 to XNpartly overlap each other on the time axis. The second embodimentassumes a possibility that two pieces of recorded data Xi and Xjselected from N pieces of recorded data do not overlap each other on thetime axis. For example, recorded data X2 in FIG. 6 partly overlapsrecorded data X1, but does not overlap recorded data X5 on the timeaxis.

Here, in a case of a recorded data pair Pij in which recorded data Xiand recorded data Xj do not partly overlap on the time axis, that is, ina case where an actual time difference of the recorded data pair Pijexceeds the time length of the recorded data Xi, the cross-correlationCij(τ) of the recorded data pair Pij does not assume a significantvalue. Hence, candidate values D identified from the cross-correlationCij(τ) are not significant values either. As is understood from FIG. 6,even in the case where the recorded data Xi and the recorded data Xj donot partly overlap each other on the time axis, when N pieces ofrecorded data X1 to XN are arranged such that the two pieces of recordeddata Xi and Xj adjacent to each other overlap each other on the timeaxis, a total value S of the N time differences O12 to ON1 over the Nrecorded data pairs P12 to PN1 is a numerical value close to zero, asdescribed in the first embodiment. However, when the N pieces ofrecorded data X1 to XN are arranged such that the two pieces of recordeddata Xi and Xj not overlapping each other on the time axis are adjacentto each other as in relation between the recorded data X2 and therecorded data X5 in FIG. 7, the total value S of the N time differencesO12 to ON1 is a numerical value deviating from zero. That is, in thesecond embodiment, the propriety of candidate values D can differaccording to the order of arrangement of the N pieces of recorded dataX1 to XN. Accordingly, in the second embodiment, the total value S ofthe candidate values D of the recorded data pairs P12 to PN1 iscalculated for each of K permutations Q1 to QK in which the order ofarrangement of the N pieces of recorded data X1 to XN is made to differ.

As in the first embodiment, a recorded data analyzing unit 40 in thesecond embodiment includes a candidate value calculating unit 42 and ananalysis processing unit 44. When the N pieces of recorded data X1 to XNare cyclically arranged in the K permutations (circular permutations) Q1to QK, the N recorded data pairs P12 to PN1 each formed by two pieces ofrecorded data Xi and Xj adjacent to each other are identified. For eachof the N recorded data pairs P12 to PN1 in each of the permutations Q1to QK, the candidate value calculating unit 42 in the second embodimentcalculates M candidate values D1 to DM for a time difference τ betweenaudio signals y(t) in the two respective pieces of recorded data Xi andXj of the recorded data pair Pij. The candidate value calculating unit42 in the first embodiment does not need to consider a plurality ofpermutations Q in calculating the candidate values D, but calculates thecandidate values D of the recorded data pair Pij only for one arbitrarypermutation. On the other hand, the candidate value calculating unit 42in the second embodiment calculates the candidate values D of eachrecorded data pair Pij for each of the K permutations Q1 to QK. As inthe first embodiment, the plurality of candidate values D are calculatedaccording to the absolute value |Cij(τ)| of the audio signalcross-correlation Cij(τ) between the recorded data Xi and the recordeddata Xj for each of the N recorded data pairs P12 to PN1.

As in the first embodiment, for each of the N recorded data pairs P12 toPN1, the analysis processing unit 44 in the second embodiment identifiesone of the M candidate values D as the time difference Oij between thetwo pieces of recorded data Xi and Xj in the recorded data pair Pij.Specifically, as in the first embodiment, for each of the K permutationsQ1 to QK, the analysis processing unit 44 calculates the total value Sof the N candidate values Dm in all combinations in which one candidatevalue Dm is selected from the M candidate values D of each recorded datapair Pij. Then, the analysis processing unit 44 identifies a combinationof candidate values Dm whose total value S is closest to zero among thetotal values S calculated for the respective combinations of candidatevalues D for each of the K permutations Q1 to QK, and determines the Ncandidate values Dm included in the combination as the respective timedifferences Oij of the N recorded data pairs Pij. As is understood fromthe above description, the analysis processing unit 44 functions as anelement that identifies one of the plurality of candidate values D ineach of the N recorded data pairs P12 to PN1 in one of the Kpermutations Q1 to QK as the time difference Oij between the two piecesof recorded data Xi and Xj in the recorded data pair Pij such that anumerical value (total value S) obtained by summing, over the N recordeddata pairs P12 to PN1, one of the plurality of candidate values Dcalculated for each of the K permutations Q1 to QK approaches zero. Asin the first embodiment, the edit processing unit 46 generates content Zin which the N pieces of recorded data X1 to XN are synchronized witheach other according to the N time differences O12 to ON1 determined bythe analysis processing unit 44.

FIG. 8 is a flowchart of processing in which the electronic controller22 generates content Z. The processing of FIG. 8 is started by beingtriggered by an instruction from a user to the operating device 36. In acase where there are, for example, four pieces of recorded data X1 to X4obtained by recording a state of a play performed on a stage frommutually different positions, when the processing of FIG. 8 is started,the candidate value calculating unit 42 determines one arbitrarypermutation Q from six permutations Q1 to Q6 in which the four pieces ofrecorded data X are cyclically arranged (SD1). For the determinedpermutation Q, as in the first embodiment, the recorded data analyzingunit 40 (the candidate value calculating unit 42 and the analysisprocessing unit 44) performs processing from the processing ofcalculating the absolute value |Cij(τ)| of the cross-correlation Cij(τ)for each of the four recorded data pairs Pij (SA1) to the processing ofcalculating a total value S of candidate values Dm (SB2).

The analysis processing unit 44 repeats the processing of steps SB1 andSB2 until the calculation of the total value S is completed for allcombinations of candidate values Dm (SB3: NO). When the total values Sof all the combinations are calculated (SB3: YES), the candidate valuecalculating unit 42 determines whether the calculation of the totalvalues S of all the combinations of the candidate values Dm (steps SA1to SB2) is completed for all the permutations Q1 to Q6 in which the fourpieces of recorded data X are arranged (SD2). When the total values Sare calculated for all the permutations Q1 to Q6 (SD2: YES), theanalysis processing unit 44 determines four candidate values Dmcorresponding to a total value S closest to zero among the total valuesS of all the combinations in all the permutations Q1 to Q6 as fourrespective time differences Oij corresponding to the four recorded datapairs Pij (SB4). When the calculation of the total values S is notcompleted for all the permutations Q1 to Q6 (SD2: NO), the candidatevalue calculating unit 42 selects an unprocessed permutation Q anew(SD1), and repeats the processing of steps SA1 to SB3. The editprocessing unit 46 generates content Z as in the first embodiment (SC1).

As is understood from the above description, the candidate valuecalculating unit 42 performs the processing of determining a permutationQ of the N pieces of recorded data X1 to XN (steps SD1 and SD2) and theprocessing of calculating the M candidate values D (steps SA1 to SA3),and the analysis processing unit 44 performs the processing ofidentifying four time differences Oij corresponding to the fourrespective recorded data pairs Pij (steps SB1 to SB3 and step SB4).

The second embodiment also achieves effects similar to those of thefirst embodiment. The second embodiment can particularly identify thetime differences Oij properly even when not all of the recording periodsof the pieces of recorded data X partly overlap each other on the timeaxis. Incidentally, the time differences Oij can be identified by theconfiguration illustrated in the second embodiment also in the caseillustrated in the first embodiment in which case the recording periodsof all the pieces of recorded data X1 to XN partly overlap each other onthe time axis.

<Modifications>

Each of the embodiments illustrated above can be modified in variousmanners. Modifications will be illustrated in the following. Two or moremodifications arbitrarily selected from the following illustrations canbe integrated with each other as appropriate within a scope where nomutual inconsistency arises.

(1) In each of the foregoing embodiments, the plurality of candidatevalues D are calculated according to the time difference τ between theaudio signals y(t) included in the respective pieces of recorded data Xiand Xj of the recorded data pair Pij. However, the signals used for thecalculation of the time difference τ are not limited to the audiosignals y(t). For example, when the audio of each of the pieces ofrecorded data X includes common utterance contents, the plurality ofcandidate values D can also be calculated by analyzing the utterancecontents of the respective pieces of recorded data X by voicerecognition, and comparing a result of the analysis between the twopieces of recorded data Xi and Xj. In addition, the plurality ofcandidate values D may be calculated by comparing (for example,calculating the cross-correlation Cij(τ)), between the two pieces ofrecorded data Xi and Xj, signals indicating temporal changes in afeature quantity (for example, pitch) extracted from the audio signalsy(t). Further, the plurality of candidate values D can also becalculated by generating signals indicating temporal changes inlightness of video included in the recorded data pair Pij, for example,from video signals representing temporal changes in the video, andcomparing the signals between the two pieces of recorded data Xi and Xj.That is, the plurality of candidate values D can also be calculated byusing signals of some kind which signals are generated from signals(audio signals and video signals) representing temporal changes of therecording target. As is understood from the above description, thesignals used to calculate the plurality of candidate values D arecomprehensively expressed as time signals representing temporal changesof the recording target (audio or video) in the two respective pieces ofrecorded data X of the recorded data pair Pij. That is, the concept ofthe time signals includes not only signals (audio signals and videosignals) representing temporal changes of the recording target itselfbut also signals representing temporal changes in feature quantities ofthe recording target (signals indirectly representing temporal changesof the recording target). However, in consideration of a tendency forthe audio signals y(t) to have small differences in temporal variationaccording to a recording condition (for example, capturing positions),the configuration of each of the foregoing embodiments using the audiosignals y(t) has an advantage of being able to identify the N timedifferences O12 to ON1 between the N pieces of recorded data X1 to XNwith high accuracy as compared with a configuration using time signalssuch as video or the like.

(2) In each of the foregoing embodiments, the plurality of candidatevalues D of the recorded data pair Pij are calculated according to thecross-correlation Cij(τ). However, the index used for the calculating ofthe plurality of candidate values D is not limited to thecross-correlation Cij(τ). For example, the plurality of candidate valuesD of the recorded data pair Pij can also be calculated according to anormalized cross-correlation. The index used for the calculation of theplurality of candidate values D is arbitrary as long as the timedifference can be calculated between time signals representing temporalchanges of the recording target in the two respective pieces of recordeddata Xi and Xj of the recorded data pair Pij.

(3) In each of the foregoing embodiments, the edit processing unit 46 isincorporated in the recorded data editing system 10. However, the editprocessing unit 46 can also be incorporated in a server device or aterminal device separate from the recorded data editing system 10. Inthis case, the recorded data editing system 10 transmits the N pieces ofrecorded data X1 to XN and the N time differences O12 to ON1 identifiedby the analysis processing unit 44 to the server device or the terminaldevice. As is understood from the above description, the recorded dataediting system 10 in each of the foregoing embodiments is anillustration of a device (that is, a recorded data analyzing device)including the recorded data analyzing unit 40 that analyzes the N timedifferences O12 to ON1 for the N pieces of recorded data X1 to XN. Editprocessing (the edit processing unit 46) is not essential in therecorded data processing device according to the some embodiments.

(4) In each of the foregoing embodiments, the time differences Oij areidentified for all the pieces of recorded data X obtained from theplurality of recording devices 12. However, it is also possible toanalyze the time differences Oij for a part of the pieces of recordeddata X obtained from the plurality of recording devices 12. For example,the recorded data analyzing unit 40 obtains time information indicatinga recording period (for example, a start time and an end time) of eachpiece of recorded data X from each of the recording devices 12 togetherwith the recorded data X, identifies N pieces of recorded data X1 to XNwhose recording times indicated by the time information overlap eachother on the time axis, and performs operation similar to that of thefirst embodiment. The candidate value calculating unit 42 functions asan element that calculates a plurality of candidate values D for each ofthe N recorded data pairs P12 to PN1 of the N pieces of recorded data X1to XN that overlap each other on the time axis among the plurality ofpieces of recorded data X. That is, recorded data X estimated not tooverlap the other recorded data X on the time axis from the timeinformation is excluded from processing targets. Incidentally, whiletemporal relation (time difference Oij) between the pieces of recordeddata X can also be identified from the time information, times measuredby the recording devices 12 can actually have errors in the respectiverecording devices 12. There is thus a meaning in identifying the timedifference Oij by the configuration in each of the foregoingembodiments. The above configuration can exclude the recorded data X notoverlapping the other recorded data X on the time axis from processingtargets, and therefore reduce a processing load on the recorded dataanalyzing unit 40 as compared with the configuration that sets all thepieces of recorded data X generated by the recording devices 12 asprocessing targets.

(5) The recorded data analyzing unit 40 illustrated in each of theforegoing embodiments is implemented by cooperation between theelectronic controller 22 and a program, as described above. The programcan be provided in a form of being stored on a recording medium readableby a computer, and installed on the computer. The recording medium is,for example, a non-transitory recording medium. An optical recordingmedium (optical disk) such as a compact disc read only memory (CD-ROM)or the like is a good example of the recording medium. However, therecording medium can include publicly known arbitrary types of recordingmedia such as a semiconductor recording medium, a magnetic recordingmedium, and the like. It is to be noted that the non-transitoryrecording medium includes arbitrary recording media excluding transitorypropagating signals, and does not exclude volatile recording media. Inaddition, the program can also be provided to a computer in a form ofdistribution via a communication network.

(6) In some embodiments can also be identified as an operating method(recorded data processing method) of the recorded data analyzing unit 40according to each of the foregoing embodiments. Specifically, in arecorded data processing method according to some embodiments, for eachof N (N is a natural number of three or more) recorded data pairs P12 toPN1 each formed by two pieces of recorded data Xi and Xj that areadjacent to each other when N pieces of recorded data X eachrepresenting a recording target including at least one of audio andvideo are cyclically arranged, a plurality of candidate values D arecalculated for a time difference between time signals representingtemporal changes of the recording target in the two respective pieces ofrecorded data Xi and Xj of the recorded data pair Pij, and one of theplurality of candidate values D in each of the N recorded data pairs Pijis identified as the time difference Oij between the two pieces ofrecorded data Xi and Xj in the recorded data pair Pij such that anumerical value obtained by summing, over the N recorded data pairs P12to PN1, one of the plurality of candidate values D calculated for eachof the N recorded data pairs P12 to PN1 approaches zero.

(7) For example, the following configurations may be understood from theembodiments exemplified above.

A recorded data processing method according to one aspect includes:calculating, for each of N (N is a natural number of three or more)recorded data pairs each formed by two pieces of recorded data that areadjacent to each other when N pieces of recorded data each representinga recording target including at least one of audio and video arearranged cyclically, a plurality of candidate values for a timedifference between time signals representing temporal changes of therecording target in the two respective pieces of recorded data of therecorded data pair; and identifying one of the plurality of candidatevalues in each of the N recorded data pairs as the time differencebetween the two pieces of recorded data in the recorded data pair suchthat a numerical value obtained by summing, over the N recorded datapairs, one of the plurality of candidate values calculated for each ofthe N recorded data pairs approaches zero. The above method identifiesone of the plurality of candidate values in each of the N recorded datapairs as the time difference between the two pieces of recorded data inthe recorded data pair such that the numerical value obtained bysumming, over the N recorded data pairs, one of the plurality ofcandidate values calculated for each of the N recorded data pairs eachrepresenting the recording target including the at least one of theaudio and the video approaches zero. With a method of adopting a soletime difference identified from between the two pieces of recorded dataas a final value, for example, a method of determining a time differencemaximizing time signal cross-correlation between the two pieces ofrecorded data as the time difference between the pieces of recordeddata, there is a possibility of an error occurring in the timedifference between the pieces of recorded data when noise is included inthe time signals. In the foregoing aspect, one of the plurality ofcandidate values is identified as the time difference between the twopieces of recorded data for each recorded data pair, and therefore thetime difference between the pieces of recorded data can be identifiedwith high accuracy. That is, it is possible to reduce an error betweenthe time difference between the pieces of recorded data and an actualtime difference.

In the recorded data processing method according to another aspect, inthe calculating of the plurality of candidate values, the plurality ofcandidate values of each of the recorded data pairs are calculatedaccording to a cross-correlation between the time signals in the twopieces of recorded data of the recorded data pair. The above methodcalculates the plurality of candidate values for the time differencebetween the two pieces of recorded data according to thecross-correlation between the time signals. Hence, the plurality ofcandidate values for the time difference between the pieces of recordeddata can be calculated simply as compared with a method of calculatingthe plurality of candidate values according to an index other than thecross-correlation.

In the recorded data processing method according to another aspect, inthe calculating of the plurality of candidate values, a plurality oftime differences corresponding to respective maxima when an absolutevalue of the cross-correlation between the time signals in the twopieces of recorded data is smoothed are identified as the plurality ofcandidate values. The above method calculates the plurality of candidatevalues for the time difference by smoothing the absolute value of thecross-correlation calculated for the two pieces of recorded dataadjacent to each other. With a method of calculating the plurality ofcandidate values for the time difference in descending order ofnumerical values of the absolute value of the cross-correlation withoutsmoothing, the plurality of candidate values can be localized within arange around a time difference maximizing the absolute value of thecross-correlation, and therefore in a case where the cross-correlationlocally increases due, for example, to noise or the like, there is apossibility of a plurality of candidate values being locally identifiedfrom the vicinity thereof. According to the foregoing method ofidentifying the plurality of time differences corresponding to therespective maxima when the absolute value of the cross-correlation issmoothed as the plurality of candidate values, the plurality ofcandidate values are dispersed. Thus, even when the cross-correlationbecomes a maximum due, for example, to noise or the like, the pluralityof candidate values can be identified so as to include a proper timedifference between the two pieces of recorded data (in turn, the timedifference can be identified with high accuracy).

In the recorded data processing method according to another aspect, thetime signals are audio signals representing temporal changes in theaudio. The above method calculates the plurality of candidate values forthe time difference between the audio signals. Hence, because timesignals representing temporal changes of video or the like have largedifferences in temporal variation according to a recording condition(for example, photographing positions), but the audio signals have smalldifferences in temporal variation according to the recording condition,there is an advantage of being able to identify time differences betweena plurality of pieces of recorded data with high accuracy.

In the recorded data processing method according to another aspect, inthe calculating of the candidate values, the plurality of candidatevalues are calculated for each of the N recorded data pairs in the Npieces of recorded data overlapping each other on a time axis among aplurality of pieces of recorded data. The above method calculates theplurality of candidate values for each of the N recorded data pairs inthe N pieces of recorded data overlapping each other on the time axisamong the plurality of pieces of recorded data. Hence, processing can beperformed which calculates the time difference of the recorded data pairafter excluding recorded data not overlapping the other recorded data onthe time axis. It is therefore possible to reduce a load of processingof calculating the time difference of the recorded data pair.

In the recorded data processing method according to another aspect, inthe calculating of the plurality of candidate values, the plurality ofcandidate values in each of the N recorded data pairs are calculated foreach of K permutations in cyclic arrangements of the N pieces ofrecorded data, and one of the plurality of candidate values in each ofthe N recorded data pairs in one of the K permutations is identified asthe time difference between the two pieces of recorded data in therecorded data pair such that a numerical value obtained by summing, overthe N recorded data pairs, one of the plurality of candidate valuescalculated for each of the K permutations approaches zero. The abovemethod identifies one of the plurality of candidate values in each ofthe N recorded data pairs in one of the K permutations as the timedifference between the two pieces of recorded data in the recorded datapair such that the numerical value obtained by summing, over the Nrecorded data pairs, one of the plurality of candidate values calculatedfor each of the K permutations approaches zero. Hence, the timedifference can be identified properly even when not all of the recordingperiods of the N pieces of recorded data partly overlap each other onthe time axis.

In the recorded data processing method according to another aspect, theN pieces of recorded data are synchronized with each other according tothe time difference identified for each of the N recorded data pairs,and content is generated from the N synchronized pieces of recordeddata. The above configuration can synchronize the N pieces of recordeddata with each other according to the time difference identified foreach of the N recorded data pairs, and generate the content from the Nsynchronized pieces of recorded data.

A recorded data processing device according to one aspect includes: acandidate calculating unit configured to, for each of N (N is a naturalnumber of three or more) recorded data pairs each formed by two piecesof recorded data that are adjacent to each other when N pieces ofrecorded data each representing a recording target including at leastone of audio and an video are arranged cyclically, calculate a pluralityof candidate values for a time difference between time signalsrepresenting temporal changes of the recording target in the tworespective pieces of recorded data of the recorded data pair; and ananalysis processing unit configured to identify one of the plurality ofcandidate values in each of the N recorded data pairs as the timedifference between the two pieces of recorded data in the recorded datapair such that a numerical value obtained by summing, over the Nrecorded data pairs, one of the plurality of candidate values calculatedfor each of the N recorded data pairs approaches zero. The aboveconfiguration identifies one of the plurality of candidate values ineach of the N recorded data pairs as the time difference between the twopieces of recorded data in the recorded data pair such that thenumerical value obtained by summing, over the N recorded data pairs, oneof the plurality of candidate values calculated for each of the Nrecorded data pairs each representing the recording target including theat least one of the audio and the video approaches zero. With aconfiguration that adopts a sole time difference identified from betweenthe two pieces of recorded data as a final value, for example, aconfiguration that determines a time difference maximizing time signalcross-correlation between the two pieces of recorded data as the timedifference between the pieces of recorded data, there is a possibilityof an error occurring in the time difference between the pieces ofrecorded data when noise is included in the time signals. In theforegoing aspect, one of the plurality of candidate values is identifiedas the time difference between the two pieces of recorded data for eachrecorded data pair, and therefore the time difference between the piecesof recorded data can be identified with high accuracy. That is, it ispossible to reduce an error between the time difference between thepieces of recorded data and an actual time difference.

What is claimed is:
 1. A recorded data processing method comprising:calculating, for each of N (N is a natural number of three or more)recorded data pairs each formed by two pieces of recorded data that areadjacent to each other when N pieces of recorded data each representinga recording target including at least one of audio and video arearranged cyclically, a plurality of candidate values for a timedifference between time signals representing temporal changes of therecording target in the two respective pieces of recorded data of therecorded data pair; and identifying one of the plurality of candidatevalues in each of the N recorded data pairs as the time differencebetween the two pieces of recorded data in the recorded data pair suchthat a numerical value obtained by summing, over the N recorded datapairs, one of the plurality of candidate values calculated for each ofthe N recorded data pairs approaches zero.
 2. The recorded dataprocessing method according to claim 1, wherein in the calculating ofthe plurality of candidate values, the plurality of candidate values ofeach of the recorded data pairs are calculated according to across-correlation between the time signals in the two pieces of recordeddata of the recorded data pair.
 3. The recorded data processing methodaccording to claim 2, wherein in the calculating of the plurality ofcandidate values, a plurality of time differences corresponding torespective maxima when an absolute value of the cross-correlationbetween the time signals in the two pieces of recorded data is smoothedare identified as the plurality of candidate values.
 4. The recordeddata processing method according to claim 1, wherein the time signalsare audio signals representing temporal changes in the audio.
 5. Therecorded data processing method according to claim 1, wherein in thecalculating of the plurality of candidate values, the plurality ofcandidate values are calculated for each of the N recorded data pairs inthe N pieces of recorded data overlapping each other on a time axisamong a plurality of pieces of recorded data.
 6. The recorded dataprocessing method according to claim 1, wherein in the calculating ofthe plurality of candidate values, the plurality of candidate values ineach of the N recorded data pairs are calculated for each of Kpermutations in cyclic arrangements of the N pieces of recorded data,and one of the plurality of candidate values in each of the N recordeddata pairs in one of the K permutations is identified as the timedifference between the two pieces of recorded data in the recorded datapair such that a numerical value obtained by summing, over the Nrecorded data pairs, one of the plurality of candidate values calculatedfor each of the K permutations approaches zero.
 7. The recorded dataprocessing method according to claim 1, wherein content in which the Npieces of recorded data are synchronized with each other according tothe time difference identified for each of the N recorded data pairs isgenerated.
 8. A recorded data processing device comprising: a candidatevalue calculating unit configured to calculate, for each of N (N is anatural number of three or more) recorded data pairs each formed by twopieces of recorded data that are adjacent to each other when N pieces ofrecorded data each representing a recording target including at leastone of audio and video are arranged cyclically, a plurality of candidatevalues for a time difference between time signals representing temporalchanges of the recording target in the two respective pieces of recordeddata of the recorded data pair; and an analysis processing unitconfigured to identify one of the plurality of candidate values in eachof the N recorded data pairs as the time difference between the twopieces of recorded data in the recorded data pair such that a numericalvalue obtained by summing, over the N recorded data pairs, one of theplurality of candidate values calculated for each of the N recorded datapairs approaches zero.
 9. The recorded data processing device accordingto claim 8, wherein in the calculating of the plurality of candidatevalues, the plurality of candidate values of each of the recorded datapairs are calculated according to a cross-correlation between the timesignals in the two pieces of recorded data of the recorded data pair.10. The recorded data processing device according to claim 9, wherein inthe calculating of the plurality of candidate values, a plurality oftime differences corresponding to respective maxima when an absolutevalue of the cross-correlation between the time signals in the twopieces of recorded data is smoothed are identified as the plurality ofcandidate values.
 11. The recorded data processing device according toclaim 8, wherein the time signals are audio signals representingtemporal changes in the audio.
 12. The recorded data processing deviceaccording to claim 8, wherein in the calculating of the plurality ofcandidate values, the plurality of candidate values are calculated foreach of the N recorded data pairs in the N pieces of recorded dataoverlapping each other on a time axis among a plurality of pieces ofrecorded data.
 13. The recorded data processing device according toclaim 8, wherein in the calculating of the plurality of candidatevalues, the plurality of candidate values in each of the N recorded datapairs are calculated for each of K permutations in cyclic arrangementsof the N pieces of recorded data, and one of the plurality of candidatevalues in each of the N recorded data pairs in one of the K permutationsis identified as the time difference between the two pieces of recordeddata in the recorded data pair such that a numerical value obtained bysumming, over the N recorded data pairs, one of the plurality ofcandidate values calculated for each of the K permutations approacheszero.
 14. The recorded data processing device according to claim 8,wherein content in which the N pieces of recorded data are synchronizedwith each other according to the time difference identified for each ofthe N recorded data pairs is generated.